RDA-SHARC’s Open Science Rewards and Incentives Survey
Author
Florencia Grattarola
Published
February 16, 2024
This survey addresses awareness of Open Science activities and existing & expected reward systems in research assessment, in particular for sharing activities, in different contexts.
Quick summary of respondents background: We received 230 responses, from people working in 37 countries, of which 41.3% declare their gender was female, 35.65% male, 1.3% non-binary or gender-queer and 21.74% had no response. Most respondents had ‘Researcher’ as their main job title (40.87%), were affiliated to a ‘University’ (39.13%) or a ‘Research institute’ (37.39%), and had between 10 and 20 years of experience in their field (32.17%).
Section 1
Are you familiar with Open Science?
Code
survey %>%group_by(are_you_familiar_with_open_science) %>%count() %>%ungroup() %>%mutate(`%`= (scales::label_percent())(n/sum(n))) %>%rename(`Are you familiar with Open Science?`= are_you_familiar_with_open_science) %>% kableExtra::kbl(booktabs = T) %>% kableExtra::kable_styling(latex_options =c("striped","hold_position"))
Are you familiar with Open Science?
n
%
No
22
10%
Yes
208
90%
Code
# generalsurvey %>%group_by(are_you_familiar_with_open_science) %>%count() %>%ggplot(aes(y = n, x ="", fill = are_you_familiar_with_open_science)) +geom_bar(stat ="identity", position ="fill") +scale_y_continuous(labels = scales::percent_format()) +scale_fill_brewer(palette ="Set1") +theme_light() +labs(fill ="Are you familiar\nwith Open Science?",x ="", y ="")
Are you involved in some of the following Open Science activities?
Code
# Select only the likert items in the# surveyitems_involved <-select(survey, starts_with("are_you_involved_in_some_of"))# Rename the items so that the question# statement becomes the namenames(items_involved) <-c("Sharing a research manuscript as a preprint","Publishing a paper or monograph book as open access","Preregistration of the study design, methods, hypothesis etc., prior to commencing the research","Open or FAIR data management and sharing (for research data, software, models, algorithms, workflows etc.)","Participation in open peer review (being reviewed or the reviewer)","Participation in public engagement, including citizen or community science","Collaboration via virtual research environments or virtual laboratories","None of the above")# We don't want this optionitems_involved <- items_involved %>%select(-`None of the above`)# A custom function to recode numerical# responses into ordered factorslikert_recode_yn <-function(x) { y <-factor(x, levels =c("No", "Yes"),exclude ="N/A")return(y)}# Transform the items into factors and# save the data set as a likert objectitems_involved <- items_involved %>%mutate_all(likert_recode_yn) %>%as.data.frame() %>% likert::likert()# percentagesitems_involved$results %>% kableExtra::kbl(booktabs = T) %>% kableExtra::kable_styling(latex_options =c("striped","hold_position")) %>% kableExtra::column_spec(1, width ="30em")
Item
No
Yes
Sharing a research manuscript as a preprint
47
53
Publishing a paper or monograph book as open access
37
63
Preregistration of the study design, methods, hypothesis etc., prior to commencing the research
84
16
Open or FAIR data management and sharing (for research data, software, models, algorithms, workflows etc.)
47
53
Participation in open peer review (being reviewed or the reviewer)
60
40
Participation in public engagement, including citizen or community science
50
50
Collaboration via virtual research environments or virtual laboratories
# generalsurvey %>%group_by(are_you_familiar_with_the_fair_findable_accessible_interoperable_and_reusable_principles_defined_here) %>%count() %>%ggplot(aes(y = n, x ="", fill = are_you_familiar_with_the_fair_findable_accessible_interoperable_and_reusable_principles_defined_here)) +geom_bar(stat ="identity", position ="fill") +scale_y_continuous(labels = scales::percent_format()) +scale_fill_brewer(palette ="Set1") +theme_light() +labs(fill ="Are you familiar\nwith the FAIR principles?",x ="", y ="")
Are you involved in some steps of data FAIRification process?
# generalsurvey %>%rename(response = does_your_institute_organisation_have_policies_on_various_open_science_activities) %>%group_by(response) %>%count() %>%ggplot(aes(x ="", y = n, fill =fct_relevel(response,"No", "Yes", "N/A"))) +geom_bar(stat ="identity",position ="fill") +scale_y_continuous(labels = scales::percent_format()) +scale_fill_brewer(palette ="Set1") +theme_light() +labs(fill ="Does your institute/\norganisation have policies\non various Open Science\nactivities?",x ="", y ="")
Section 2
Could you please specify to what extent you feel the following activities should be credited / rewarded?
Rewards may include career promotion, grants/funding/prizes, gained credits in a research evaluation procedure, authorship/ contributorship, increased academic visibility.
Code
# Select only the likert items in the# surveyitems <-select(survey, starts_with(c("could_you_please_specify_to_what_extent")))# Rename the items so that the question# statement becomes the namenames(items) <-c("Sharing a research manuscript as a preprint","Publishing a paper or monograph book as open access","Preregistration of the study design, methods, hypothesis etc., prior to commencing the research","Open or FAIR data management and sharing (for research data, software, models, algorithms, workflows etc.)","Participation in open peer review (being reviewed or the reviewer)","Participation in public engagement, including citizen or community science","Collaboration via virtual research environments or virtual laboratories")# A custom function to recode numerical# responses into ordered factorslikert_recode <-function(x) { y <-factor(x, levels =c("Definitely Not","Probably Not", "Possibly", "Very Probably","Definitely"), exclude ="No opinion")return(y)}# Transform the items into factors and# save the data set as a likert objectitems_likert <- items %>%mutate_all(likert_recode) %>%as.data.frame() %>% likert::likert()# percentagesitems_likert$results %>% kableExtra::kbl(booktabs = T) %>% kableExtra::kable_styling(latex_options =c("striped","hold_position")) %>% kableExtra::column_spec(1, width ="10em")
Item
Definitely Not
Probably Not
Possibly
Very Probably
Definitely
Sharing a research manuscript as a preprint
4.21
10.3
31
27
27
Publishing a paper or monograph book as open access
0.89
4.5
16
23
55
Preregistration of the study design, methods, hypothesis etc., prior to commencing the research
3.35
9.1
29
31
28
Open or FAIR data management and sharing (for research data, software, models, algorithms, workflows etc.)
0.90
2.7
14
26
56
Participation in open peer review (being reviewed or the reviewer)
1.81
6.3
30
27
35
Participation in public engagement, including citizen or community science
0.90
4.1
26
24
45
Collaboration via virtual research environments or virtual laboratories
# generalsurvey %>%rename(response = does_your_institute_organisation_have_any_initiative_or_tool_which_gives_credits_rewards_for_open_science_activities) %>%group_by(response) %>%count() %>%ggplot(aes(y = n, x ="", fill = response)) +geom_bar(stat ="identity", position ="fill") +scale_y_continuous(labels = scales::percent_format()) +scale_fill_brewer(palette ="Set1") +theme_light() +labs(fill ="Does your institute/organisation\nhave any initiative or tool which\ngives credits/rewards for\nOpen Science activities?",x ="", y ="")
Code
# by countrysurvey %>%rename(response = does_your_institute_organisation_have_any_initiative_or_tool_which_gives_credits_rewards_for_open_science_activities,country = which_country_do_you_work_in) %>%group_by(response, country) %>%count() %>%filter(response =="Yes") %>%ggplot(aes(y = n, x = country, fill = country)) +geom_bar(stat ="identity") +scale_fill_brewer(palette ="Spectral") +scale_fill_manual(values =colorRampPalette(RColorBrewer::brewer.pal(9,"Set1"))(15)) +theme_light() +theme(axis.text.x =element_text(size =rel(0.5))) +labs(caption ="Mentioned initiative or tool which gives credits/rewards for Open Science activities by country",fill ="", x ="", y ="")
Section 4
How would you want the previously mentioned Open Science activities to be rewarded?
Labels
Code
rewardLabels <- survey %>%rename(labels = suggested_rewards_categorised_according_to_shar_cs_rewarding_terminology) %>%separate_rows(labels, sep =";") %>%mutate(labels =str_squish(labels)) %>%filter(!is.na(labels) & labels !="") %>%mutate(labels =ifelse(labels =="good science'","'good science'", labels)) %>%mutate(labels =ifelse(labels =="funding/grants for OS activitie","funding/grants for OS activities", labels)) %>%mutate(labels =ifelse(labels =="OS indicators in research evaluation processes","OS indicators in research evaluation and/or career progression processes", labels))rewardLabels %>%group_by(labels) %>%count() %>%arrange(desc(n)) %>%ungroup() %>%mutate(`%`= (scales::label_percent())(round(n/sum(n),3))) %>%rename(`SHARC Rewards Terminology`= labels) %>% kableExtra::kbl(booktabs = T) %>% kableExtra::kable_styling(latex_options =c("striped","hold_position"))
SHARC Rewards Terminology
n
%
OS indicators in research evaluation and/or career progression processes
126
40.3%
funding/grants for OS activities
49
15.7%
awards/bonuses
33
10.5%
research visibility indicators
26
8.3%
authorship/contributorship
20
6.4%
capacity building for OS (e.g., training, raising awareness, provision of IT tools)
contribution to ‘good’ science, research quality and integrity
3
1.0%
OS certifications/badges
2
0.6%
science as a public good
2
0.6%
CRediT taxonomy
1
0.3%
OS activities included in working hours
1
0.3%
championships/contests
1
0.3%
financial contribution to reviewers of OA journals
1
0.3%
punishment for 'closed' science
1
0.3%
Code
rewardLabels %>%filter(!is.na(labels)) %>%group_by(labels) %>%count() %>%ggplot(aes(x ="", y = n, fill =fct_reorder(labels, n))) +geom_bar(stat ="identity",position ="fill") +guides(fill =guide_legend(ncol =1)) +scale_y_continuous(labels = scales::percent_format()) +scale_fill_manual(values =colorRampPalette(RColorBrewer::brewer.pal(9,"Set1"))(19)) +theme_light() +labs(fill ="SHARC Rewards\nTerminology",x ="", y ="") +theme(legend.position ="right")
Code
most_common_rewards <-c("OS indicators in research evaluation and/or career progression processes","funding/grants for OS activities", "awards/bonuses","research visibility indicators", "authorship/contributorship")rewardLabels %>%filter(!is.na(labels)) %>%mutate(labels =ifelse(!labels %in% most_common_rewards,"others", labels)) %>%group_by(labels) %>%count() %>%ggplot(aes(x ="", y = n, fill =fct_relevel(fct_reorder(labels, n), "others"))) +geom_bar(stat ="identity",position ="fill") +guides(fill =guide_legend(ncol =1)) +scale_y_continuous(labels = scales::percent_format()) +scale_fill_brewer(palette ="Set1") + ggpubr::theme_cleveland() +labs(fill ="SHARC Rewards\nTerminology",x ="", y ="") +theme(legend.position ="right")
Respondents’ background
Gender
These are the different responses and how they were classified. I now added ‘non-binary’ as a different category and tagged all other as ‘other responses’.
# which_country_do_you_work_insurvey %>%rename(response = which_country_do_you_work_in) %>%group_by(response) %>%count() %>%ggplot(aes(x ="", y = n, fill =fct_reorder(response,desc(n)))) +geom_bar(stat ="identity") +scale_fill_manual(values =colorRampPalette(RColorBrewer::brewer.pal(9,"Set1"))(37)) +theme_light() +labs(caption ="Which country do you work in?",fill ="", x ="", y ="")
Code
# Korea, France, USA, Uruguay, and# Other countriessurvey %>%rename(response = which_country_do_you_work_in) %>%mutate(response_x =ifelse(response !="Korea, Rep."& response !="France"& response !="United States"& response !="Uruguay", "Other countries", response)) %>%group_by(response_x) %>%count() %>%ggplot(aes(x ="", y = n, fill =fct_relevel(fct_reorder(response_x, n), "Other countries")), show.legend =FALSE) +geom_bar(stat ="identity", position ="fill") +scale_y_continuous(labels = scales::percent_format()) +scale_fill_brewer(palette ="Spectral") +labs(fill ="") +theme_light() +labs(caption ="Which country do you work in?",x ="", y ="")
survey %>%group_by(how_long_have_you_been_working_in_your_field) %>%count() %>%ungroup() %>%mutate(`%`= (scales::label_percent())(round(n/sum(n),3))) %>%rename(`Experience in the field`= how_long_have_you_been_working_in_your_field) %>%mutate(index =ifelse(`Experience in the field`=="> 20 years", 1, ifelse(`Experience in the field`=="10-20 years", 2, ifelse(`Experience in the field`=="5-10 years", 3, ifelse(`Experience in the field`=="< 5 years", 4, 5))))) %>%arrange(index) %>%select(-index) %>% kableExtra::kbl(booktabs = T) %>% kableExtra::kable_styling(latex_options =c("striped","hold_position"))
Experience in the field
n
%
> 20 years
61
26.5%
10-20 years
74
32.2%
5-10 years
56
24.3%
< 5 years
38
16.5%
NA
1
0.4%
Code
survey %>%rename(response = how_long_have_you_been_working_in_your_field) %>%group_by(response) %>%count() %>%filter(!is.na(response)) %>%ggplot(aes(x ="", y = n, fill =fct_relevel(response,"> 20 years", "10-20 years", "5-10 years","< 5 years"))) +geom_bar(stat ="identity",position ="fill") +scale_y_continuous(labels = scales::percent_format()) +scale_fill_brewer(palette ="Spectral") +theme_light() +labs(fill ="Years of experience",x ="", y ="")
Affiliation type of organisation
Please select the type of organisation you are primarily affiliated with